Anomalous resonant production of the fourth family up type quarks at the LHC 
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Considering the present limits on the masses of fourth family quarks from the Tevatron experiments, the 
fourth family quarks are expected to have mass larger than the top quark. Due to their expected large mass they 
could have different dynamics than the quarks of three families of the Standard Model. The resonant production 
of the fourth family up type quark t' has been studied via anomalous production subprocess gqj — > t' (where 
qi = u,c) at the LHC with the center of mass energy 10 TeV and 14 TeV. The signatures of such process are 
discussed within the SM decay modes. The sensitivity to anomalous coupling k/A = 0.1 TeV -1 can be reached 
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5^ I. INTRODUCTION 

The number of fermion families in nature, the pattern of fermion masses and the mixing angles in the quark/lepton sectors 
are two of the unanswered questions in the Standard Model (SM). The repetition of quark and lepton families remain a mystery 
as part of the flavor problem. On theoretical grounds, the asymptotic freedom in the quantum chromodynamics imposes an 
' indirect bound for the number of the quark flavors which should be less than eighteen. The electroweak precision measurements 
_-, . done by the Large Electron Positron (LEP) experiments imply that the number of light neutrinos (m v < 45 GeV) is equal to 
three [1]. The most recent analyses indicate that an additional family of heavy fermions is not inconsistent with the precision 
electroweak data at the available energies [2-8]. Indeed, the presence of three or four fermion families are equally consistent 
with the electroweak precision data, moreover the four families scenario is favored if the Higgs boson heavier than 200 GeV [4] . 
Q\ [ The fourth family may play an important role in our understanding of the flavor structure of the SM. The flavor democracy is 
■ one of the main motivations for the existence of the fourth family fermions [9]. Another motivation for the fourth family comes 
from the charge-spin unification [10]. Additional fermions can also be accommodated in many models beyond the SM [1 1]. A 
recent review of the fourth SM family including the theoretical and experimental aspects can be found in [12]. 

A lower limit on the mass of the fourth family quark Q' is nig > 256 GeV from Tevatron experiments [13], whereas the upper 
limit from partial wave unitarity is about 1 TeV. The recent results from the Collider Detector at Fermilab (CDF) experiment 
exclude the t mass below 311 GeV at 95% CL using thedataof2.8fb _1 [14]. 

The tree level flavor changing processes occur only via the charged current interactions in the SM. The first two rows of the 
Cabibbo-Kobayashi-Maskawa (CKM) matrix [15] are in good agreement with the unitary condition. The data for the number 
of b— jets in the top quark pair production at Tevatron constrain the ratio R = |^i>| 2 /Eg \Vtq\ 2 , which is closely related to V t b if 
the CKM is unitary. The direct constraints on \V t b\ come from the single production of top quarks at the Tevatron. From the 
average cross section a = 3.7 ±0.8 pb the lower limit \V t b\ > 0.74 at 95% CL is given by the CDF and D0[16]. A measurement 
of the single top production cross section smaller than the SM prediction would imply V t i, < 1 or the evidence of extra families 
of quarks mixed with the third generation. A flavor extension of the SM, with a fourth generation of quarks, leads to an extended 
CKM matrix which could have V t b smaller than 1 . In the extended model, the strongest constraint to V t b comes from the ratio 
Rb = r(Z — > bb) /T(Z — > hadrons), with R > 0.9 for m,i > 1.5m t . For an extra up type quark t' and another extra down type 
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quark b' , the 4x4 matrix is unitary, for which any 3x3 submatrix becomes non-unitary as long as these new quarks mix with 
quarks of three families. Hence, the new flavor changing neutral currents (FCNC) could appear without violating the existing 
bounds from current experimental measurements [17, 18]. 

The existence of a fourth generation of quarks would have interesting implications. Taking into account the current bounds 
on the mass of the fourth family quarks [1], the anomalous interactions can emerge in the fourth family case. Furthermore, 
extra families will yield an essential enhancement in the Higgs boson production at the LHC [19]. Single production [20], [21] 
mechanism of fourth family quarks will be suppressed by the elements (fourth row and/or fourth column) of the 4x4 CKM 
matrix. The fourth family quark pairs can already be produced at the Large Hadron Collider (LHC) at an initial center of mass 
energy of y/s = 10 TeV and an initial luminosity of L = 10 31 cm~ 2 s~'. At the nominal center of mass energy y/s = 14 TeV 
the initial luminosity will be 10 33 cm _2 s _1 which will later increase to 10 34 cm _2 s _1 corresponding to 10 and 100 fb _1 per year, 
respectively. 

In this work, we present an analysis of the anomalous resonant production of t ' quarks at the LHC. Here, we assume the case t ' 
decays through SM dominated channel (via charged currents) in which the magnitude of V t / q is important, leading to a final state 
W ± bj et for t' anomalous production. A fast simulation is performed for the detector effects on the signal and background. Any 
observations of the invariant mass peak in the interval 300-800 GeV with the final state containing W ± bj et can be interpreted as 
the signal for t' anomalous resonant production. 



n. FOURTH FAMILY QUARK INTERACTIONS 

Fourth family quarks can couple to charged weak currents by the exchange of W ± boson, neutral weak currents by Z° boson 
exchange, electromagnetic currents by photon exchange and strong colour currents by the gluons. We include the fourth family 
quarks in the enlarged framework (primed) of the SM. The interaction lagrangian is given by 

L' =-g e I QeiQ/rQ'Av-gs E QlT a f^Gl 

Q'i=h',t' Q'i=b'..t' 

where g e is the electromagnetic coupling constant, g s is the strong coupling constant. The vector fields A^, G^, and 

denote photon, gluon, Z°— boson and W ± — boson, respectively. Q e i is the electric charge of fourth family quarks, T a are the 
Gell-Mann matrices. The gv and gA are the couplings for vector and axial-vector neutral currents. Finally, the CKM matrix 
elements Vy are expressed as: V = V U V D ^ . The corresponding 4x4 CKM matrix is given by 
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The magnitude of the 3 x 3 CKM matrix elements are determined from the low energy and high energy experiments: these 
are \V ud \ = 0.97418 ± 0.00027, |V MJ | = 0.2255 ± 0.0019, \V cd \ = 0.230 ± 0.01 1, \V CS \ = 1.04 ±0.06, \V cb \ = 0.0412 ±0.0011, 
\V ub \ = 0.00393 ±0.00036, \V td \ = 0.0081 ±0.0006, \V ts \ = 0.0387 ±0.0023 (assuming \V tb \ equal to unity) and a lower limit 
from the single top production \V tb \ > 0.74 at 95% CL. [1]. In the fourth family case, the first three rows of this matrix are 
calculated as \V ub >\ 2 = 0.0008, \V cb >\ 2 = 0.0295 and \V,y\ 2 = 0.4054. For the first three columns one calculates |V ; / rf | 2 = 0.001, 



|V ; / i | 2 = 0.0315 and IV^I 2 = 0.4053. We see that there is a loose constraint for the mixing between third and fourth family 
quarks. In this case, these bounds can be relaxed to an uncertainty level. If there is a mass degeneracy between t' and b' quarks, 
the two body decays occur most probably into the third family quarks. Inspiring from the Wolfenstein parametrization of the 
3x3 CKM matrix, we could simply consider the fourth row and fourth column of the 4 x 4 CKM as \V qi y \ ~ \V,t q . \ = Aj jX 4 ~ n 
where A j; - can be optimized for the quark flavors and qj; n is the family number and A is a constant. 
We consider the decay width of t ' quark through t' — > W + q including the final state quark mass, we find 



no! 



1 a e \V &q \ 2 ml, 

' Wq > = 7Z - T77 — — X w 
lo sin Owm^ 



(3) 
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where 



A r = 1 + m% I nig, + m q / nig, — 2m 2 v / nig, — 2m 2 / nig, — 2m 2 v m 2 / nig, (4) 

Aw = 1 + m\ /Mq - 2m 2 ,/ nig, + m A q / nig, + m 2 q m\ / nig, - 2m w / m Q , (5) 

To calculate the decay width numerically, we assume three parametrizations PI, PII and PIII for the fourth family mixing 
matrix elements. For the PI parametrization we assume the constant values \Vgi q \ = \V q g/ \ = 0.01, PII contains a dynamical 
parametrization \V q .g/ \ = \Vg/ q . | ~ A 4- " with a preferred value of A — 0. 1 , and PIII has the parameters \V t i d \ = 0.063, | V t / S \ = 0.46, 
\V t , b \ = 0.47, \V u y\'= 0.044, \V c y\ = 0.46, \V tV \ - 0.47 [22]. 

The flavor changing neutral current interactions are known to be absent at tree level in the SM. However, the fourth family 
quarks, being heavier than the top quark, could have different dynamics than other quarks and they can couple to the FCNC 
currents leading to an enhancement in the resonance processes at the LHC. Moreover, the arguments for the anomalous interac- 
tions of the top quark given in [23], are more valid for t' and b' quarks. The effective Lagrangian for the anomalous interactions 
among the fourth family quarks t 1 and b', ordinary quarks q, and the neutral gauge bosons V = y,Z,g can be written explicitly: 

qj=u,c,t q i= u,cf 

+ E 1 $rgsfo^X a q i G^+h.c. 
m=u,c.t ZA 

+ E ^Q^^ vqi F^+ £ ^V<7«Z" V 

q i= d,s,b A q i= d,s,b ZJV 

+ e f?^^vM,cr+/ I . c . (6) 

qi=d,s,b 

where F^ v , Z^ v and G^ v are the field strength tensors of the gauge bosons; cj M v = '(7m 7v — 7v7^)/2; X a are the Gell-Mann 
matrices; Q q is the electric charge of the quark (q); g e , gz and g s are the electromagnetic, neutral weak and the strong coupling 
constants, respectively, gz = g e / cos vv sin0„,, where W is the weak angle. Ky is the anomalous coupling with photon; K z is for 
the Z boson, and Kg with gluon. A is the cut-off scale for the new interactions. 

For the decays Q 1 — > Vq where V = J,Z,g, we use the effective Lagrangian to calculate the anomalous decay widths 

2 /k* n 2 



r(e'^s</) = - -f a. s m 3 e ,Ao (7) 



r (2'^r?) = ^(^) a e Q 2 m Q ,Xo (8) 



16 \ A / sin z 0w cos 2 dw 



with 



Ao = 1 - 3m 2 q /m 2 Q , + 3m 4 q /m Q , - m 6 q /m Q , ( 1 0) 

Az = 2 — m z /m 2 g, — Am 2 q /m 2 gi +2m q /mg, — 6mqm z /mg, — m z m 2 /nig, — m^/nig, (11) 

The anomalous decay widths in different channels are proportional to A -2 , and they become to contribute more k/A > 0.1 
TeV- 1 . 
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Table I: Decay widths and branching ratios (%) of the fourth family quarks in case both chiral and anomalous interactions, where qi denotes 
the quarks belong to i th family. Vq\2 denotes Vq\ and Vq2- The parametrization for the 4x4 CKM elements is taken as PI, PII, PHI given in 
the text. In the last row the total decay widths of fourth family t' quarks are given for three mass values and k/A = 1 TeV -1 . The numbers in 
the paranthesis denote the values for k/A = 0.1 TeV -1 . 





PI 


PII 


PHI 


ny(GeV) 


300 


500 


700 


300 


500 


700 


300 


500 


700 


Wqi 


0.017(1.6) 


0.014(1.4) 


0.014(1.3) 


0.0002(0.0062) 


0.0001(0.0059) 


0.0001(0.0058) 


0.39(0.9) 


0.35(0.9) 


0.34(0.89) 


Wq 2 


0.017(1.6) 


0.014(1.4) 


0.014(1.3) 


0.017(0.62) 


0.014(0.59) 


0.014(0.58) 


21.0(48) 


19.0(48) 


18.0(48) 


Wq 3 


0.017(1.6) 


0.014(1.4) 


0.014(1.3) 


1.7(62) 


1.4(59) 


1.4(58) 


21.0(50) 


20.0(50) 


19.0(50) 


z 1l,2 


2.5(2.3) 


2.3(2.2) 


2.2(2.1) 


2.4(0.91) 


2.3(0.93) 


2.2(0.93) 


1.4(0.033) 


1.4(0.036) 


1.4(0.037) 




0.27(0.26) 


1.4(1.4) 


1.8(1.7) 


0.27(0.1) 


1.4(0.59) 


1.8(0.75) 


0.16(0.0036) 


0.89(0.023) 


1.1(0.03) 


m,2 


0.9(0.86) 


0.76(0.73) 


0.72(0.69) 


0.89(0.33) 


0.75(0.31) 


0.71(0.3) 


0.52(0.012) 


0.47(0.012) 


0.45(0.012) 


m 


0.26(0.25) 


0.52(0.5) 


0.6(0.57) 


0.26(0.097) 


0.51(0.21) 


0.59(0.25) 


0.52(0.0035) 


0.32(0.008) 


0.37(0.0098) 


811,2 


40(39) 


34(33) 


32(31) 


40(15) 


34(14) 


32(14) 


23.0(0.54) 


21.0(0.53) 


20.0(0.53) 


813 


12(11) 


23(22) 


27(26) 


12(4.4) 


23(9.4) 


26(11) 


6.8(0.16) 


14.0(0.36) 


20.0(0.44) 


r„„(GeV) 


5.21(0.055) 


28.47(0.297) 


82.58(0.859) 


5.298(0.141) 


28.871(0.701) 


83.71(1.97) 


9.05(3.89) 


46.46(18.29) 


132.04(50.30) 



Table II: Decay widths and branching ratios (%) of the fourth family quarks with only anomalous interactions, where qi denotes the quarks 
belong to i' h family. Vqi,2 denotes Vq\ and Vq2- In the last row the total decay widths of fourth family t' and b' quarks are given for three 
mass values and k/A=\ TeV -1 . 





t' 


Mass (GeV) 


300 


500 


700 


Z?l,2 


2.5 


2.3 


2.2 


Zq 3 


0.27 


1.4 


1.8 


711.2 


0.9 


0.76 


0.72 


m 


0.26 


0.52 


0.6 


811,2 


40.0 


34.0 


32.0 


813 


12.0 


23.0 


27.0 


r (0( (GeV) 


5.21 


28.4 


82.56 



The total decay width of t' quark is shown in Fig.l. In Table I, we give the numerical values of the total decay width and 
branching ratios for the parametrizations PI-PIII. As it can be seen from these tables, for the mass range of m t i relevant to 
LHC experiments, the fraction of the anomalous decay modes are 99.9%(95 - 96%), 98%(37 - 41%) and 58 - 62%(99%) at 
k/A = 1(0.1) TeV -1 for the parametrizations PI, PII and PHI, respectively. However, SM decay modes of t' become dominant 
at PII and PHI parametrizations when k/A = 0. 1 TeV~ 1 . For the parametrization PHI, the SM decay mode and anomalous decay 
mode become comparable for k/A = 1 TeV -1 . 

If the CKM unitarity is strictly applied and the Q' mixing with light quarks is strongly constrained, which corresponds to 
\V t i qi | ~ for the light quarks q t , there still remains room for the SM decays t ' — > W + b and the possible anomalous decays 
t' — > Vqt. In this case, f' can have significant FCNC couplings. When the anomalous interactions much dominate over the SM 
decays, still allowing only the non-vanishing element \V t iy | ~ 1 (but assuming t' and b' are mass degenerate) the decay widths 
and branching ratios of the fourth family quarks with the anomalous interactions are shown in Table II. Taking the anomalous 
coupling k/A = 0. 1 TeV -1 , we calculate the t' anomalous decay width T = 8.79 x 10 -2 GeV, r = 3.86 x 10" 1 GeV and T = 1 .02 
GeV for m t < = 350, 550 and 750 GeV, respectively. 

III. ANOMALOUS RESONANT PRODUCTION OF t ' QUARKS 

In order to study the resonant production of fourth family quarks, we have implemented the anomalous interaction vertices 
with the new particles t' into CompHEP package [24]. In all numerical calculations, the parton distribution functions (PDF) 
fq( x , Q 2 ) an d fg(y, Q 2 ) are set to tne CTEQ6M parametrization [25] and the factorization scale Q 2 = Mq, is used. The total cross 
section for the process pp — » W + bX is given by 
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Figure 1: Decay width of /' quark depending on its mass for different V t i„ parametrizations and k/A values. 
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Figure 2: Diagrams for the gq, — > W + g/ subprocess including <2'</f; (where 2' = anomalous vertices; can be any of the quarks inside the 
proton, while qj can be any of the quark flavour depending on the charged current interaction. For the W~qj final states we may change the 
direction of the current lines and replace the incoming anti-quarks. 



rl f 1 dx T 
a= dx —f q (x,Q 2 )f g (-,Q 2 )a(Ts) (12) 

JXmin Jz x x 

where S = zs and T/, m >, = (my +m q ) 2 /s; d is the partonic cross section for a given process. We consider the production qg — > t' 
and the decay t — > W + bj. In the generic notation the contributing Feynman diagrams are shown in Fig. 2. 

The production cross sections as a function of fourth family quark mass for the different parametrization are shown in Tables 
III-V. The ratios of the cross sections for different parametrizations are calculated as k c ps 0.01 for PI and k a s» 15 for PHI with 
the normalization to PII with k/A. = 1 TeV~\ as shown in Fig. 3. For the parametrization PHI we find the t'(t ) production cross 
sections 46.4(1 1.7) pb for k/A — 0.1 TeV -1 and m t i = 400 GeV at y/s = 14 TeV. The t production cross section is lower than t' 
production with a factor of 2-8 depending on the considered mass range. The general behaviour of the cross sections depending 
on the mass is presented in Figs.4-6. 



IV. SIGNAL AND BACKGROUND 

The resonant production mechanisms of the fourth family t' quarks depends on the anomalous coupling k/A, while their 
anomalous decays and charged current decays depend on both these couplings and the 4x4 CKM matrix elements. The signal 
process pp — > W + bX includes t' exchange in the s-channel. The s— channel contribution to the signal process would manifest 
itself as resonance around the t' mass value in the W-boson+jet reconstructed invariant mass. When we consider the leptonic 
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Figure 3: The ratio of the cross sections for different parametrizations normalized to PII, where k/A = 1 TeV 



Table III: Resonance cross sections (pb) and decay widths of t' quarks for PI parametrization with k/A = 1 TeV 1 and k/A = 0.1 TeV 1 at 
s/s = 14(10) TeV. 





k/A= 1 TeV" 1 


ic/A = 0.1 TeV" 1 


ny(GeV) 


CT(f') 


a(P) 


rV(GeV) 




o(P) 


rV(GeV) 


300 


2.55(0.16) 


0.744(1.73xl0~ 3 ) 


5.21 


2.53(0.149) 


0.693(1.65xl0~ 2 ) 


0.055 


400 


1.43(6.11xl0" 2 ) 


0.360(5. 16xl0- 3 ) 


13.74 


1.34(0.063) 


0.341(5.02xl0- 3 ) 


0.143 


500 


0.903(2.68xl0~ 2 ) 


0.198(1.81xl0^ 3 ) 


28.46 


0.856(0.027) 


0.192(1.77xl0~ 3 ) 


0.296 


600 


0.608(1.26xl0~ 2 ) 


0.119(7.07xl0~ 4 ) 


50.91 


0.582(0.013) 


0.114(6.84xl0~ 4 ) 


0.530 


700 


0.429(6.25xl0~ 3 ) 


0.075(3.0xl0~ 4 ) 


82.59 


0.415(0.006) 


0.075(2.84xl0~ 4 ) 


0.859 


800 


0.311(3.2xl0^ 3 ) 


0.049(1.38xl0~ 4 ) 


125.06 


0.305(0.003) 


0.051(1.22xl0~ 4 ) 


1.30 


900 


0.232(1.69xl0- 3 ) 


0.034(6.82xl0- 5 ) 


179.82 


0.217(1.7xl0- 3 ) 


0.035(5.38xl0- 5 ) 


1.86 


1000 


0.174(9.24xl0~ 4 ) 


0.023(3.66xl0~ 5 ) 


248.43 


0.178(8. 8xl0~ 4 ) 


0.025(2.39xl0~ 5 ) 


2.58 



W-decays, the f' signal search will be l + +bj et + fLj, where £ = e,fi. For the hadronic W-decays one would seek the events 
with one b-jet alongside two more jets requiring these to have an invariant mass peak around the W-mass. If we consider the 
dominance of the SM decay mode over the anomalous decay, the t' resonant production signal will be 

PP^ f(?) X 

^ W+H b jet (13) 
^ j 



Table IV: Resonance cross sections (pb) and decay widths of /' quarks for PII parametrization with k/A = 1 TeV 'and k/A = 0.1 TeV 1 at 
v/j= 14(10) TeV. 





k/A= 1 TeV~' 


K-/A = 0.1 TeV- 1 


«v(GeV) 




<y(f) 


T,'(GeV) 




e(f) 


I>(GeV) 


300 


250.92(15.48) 


73.30(1.71) 


5.29 


93.24(5.87) 


27.51(0.65) 


0.14 


400 


141.24(6.02) 


35.53(0.52) 


13.94 


55.62(2.44) 


14.18(0.21) 


0.35 


500 


88.89(2.64) 


19.61(0.18) 


28.87 


36.26(1.12) 


8.18(7.48xl0~ 2 ) 


0.70 


600 


60.00(1.25) 


11.79(6.98xl0~ 2 ) 


51.61 


25.16(0.55) 


5.06(2.95xl0~ 2 ) 


1.23 


700 


42.33(0.62) 


7.46(2.96xl0- 2 ) 


83.71 


18.14(0.28) 


3.29(1.23xl0~ 2 ) 


1.97 


800 


30.77(0.32) 


4.92(1. 36xl0~ 2 ) 


126.72 


13.45(0.14) 


2.23(5.35xl0~ 3 ) 


2.96 


900 


22.84(0.17) 


3.34(6.76xl0- 3 ) 


182.18 


10.22(0.075) 


1.55(2.37xl0~ 3 ) 


4.23 


1000 


17.23(9.1xl0~ 2 ) 


2.32(3.6xl0~ 3 ) 


251.67 


7.92(0.039) 


l.ll(1.07xl0 -3 ) 


5.82 



10 1 
10° 

10- 1 

10- 2 
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Table V: Resonance cross sections (pb) and decay widths of t quarks for PHI parametrization with k/A ■■ 
s/s= 14(10) TeV. 



lTeV _1 and k/A = 0.1 TeV" 1 at 





K/A = 1 TeV" 1 


fc/A = 0.1 TeV" 1 


m^(GeV) 




o(P) 


r f /(GeV) 


a [t') 


a(P) 


rV(GeV) 


300 


JZ, 1 Z,. \J\J\ l"Q. 1 Z. ) 


QA(S SA(1\ Q?1 
7tu. jt-^z, i .yz, ) 








j.oy 


400 


1912.50(79.96) 


475.42(6.75) 


22.92 


46.43(2.00) 


11.74(0.169) 


9.33 


500 


1228.70(35.57) 


267.17(2.42) 


46.46 


30.89(0.93) 


6.87(6.26xl0~ 2 ) 


18.28 


600 


835.63(16.85) 


161.31(0.96) 


82.03 


21.60(0.46) 


4.28(2.53xl0~ 2 ) 


31.65 


700 


590.53(8.34) 


102.24(0.42) 


132.04 


15.61(0.23) 


2.78(1. Ixl0~ 2 ) 


50.30 


800 


428.19(4.27) 


67.04(0.20) 


198.88 


11.56(0.12) 


1.87(4.95xl0- 3 ) 


75.12 


900 


315.96(2.26) 


45.14(9.96xl0~ 2 ) 


284.95 


8.73(0.064) 


1.29(2.39xl0~ 3 ) 


106.99 


1000 


236.33(1.25) 


30.94(5.53xl0~ 2 ) 


392.65 


6.69(0.035) 


0.92(1.23xl0~ 3 ) 


146.79 








^/A = 0.1TeV _l 
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Figure 4: The cross section for the anomalous production of t' quarks for PI parametrization and k/A -■ 
followed by the decay W + b(W~b) at the center of mass energy 14 TeV (left) and 10 TeV (right). 



: 0.1 TeV" 1 . The lines denotes t'(P) 



which includes further leptonic or hadronic decays of W + boson. In the second and third lines the elements of 4 x 4 CKM matrix 
Vfij, and V t ig enter to the decay process for t' signal. The cross section for the SM process pp — ► W + bX (pp^ W~bX) is 10. 14 pb 
(9.78 pb) without any cuts at 14 TeV, and 5.73 pb (5.49 pb) at 10 TeV. For the cross section estimates, we assume the efficiency 
for ^-tagging as Ey = 50%, and rejection factors rj = 100 for light jets, and r c = 10 for c(c) quark jets since they are assumed 
to be mis-tagged as b-jets. The pj distributions for both signal and background are given in Fig. 7 including the interference 
terms. Moreover, different background processes contributing to the same final state are presented in Table VI with various pj 
cuts. At the center of mass energy y/s = 10 TeV, the background cross sections are calculated as 0.375 pb (0.354 pb) for W + b 
(W~b), and 2.69 x 10 3 pb (1.88x 10 3 pb) for W + j (W~j) with p J T > 50 GeV. The invariant mass distributions of biv system 
at partonic level for different parametrizations are given in Fig. 8. Here, the W + b and W + j backgrounds are included with the 
assumed efficiencies and acceptance factors. In Fig. 9, we show a density plot of the transverse momentum of the b quark and 
the invariant mass distribution of W + b system with the t' mass constraint. 



V. ANALYSIS 



At the generator level, we have required a fo-jet with transverse momentum at least p b T > 50 GeV for the W + b events. The 
events generated for each subprocess are mixed using the "mix" script which can be found in the CompHEP package [24], 
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Figure 5: The cross section for the anomalous production of t' quarks for PII parametrization and fc/A = 0.1 TeV . The lines denote r'(f') 
production followed by the decay W + b(W~b) at the center of mass energy 14 TeV (left) and 10 TeV (right). 
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Figure 6: The cross section for the anomalous production of /' quarks for PHI parametrization and k/A = 0.1 TeV 
production followed by the decay W + b(W~b) at the center of mass energy 14 TeV (left) and 10 TeV (right). 



The lines denote f'(f') 



and passed to the PYTHIA [26] for further decays and hadronization using the cpyth package [27]. After W -boson decay and 
hadronization, the detector effects, such as acceptance and resolution are simulated with PGS4 program [28] using generic 
LHC detector [29] parameters. This fast simulation includes the most important detector effects, such as pj smearing, E em 
and Ef,ad smearing, energy deposited in towers (granularity) and tag efficiencies of the remaining detector effects such as mis- 
identifications. More realistic simulation requires the resources of the LHC Collaborations which are beyond the scope of this 
work. ExRoot Analysis package [30] is used to PGS4 data and the output is analysed and histogrammed with the ROOT [31] 
macros. Since the cross section for t' — ► W + b is about five times larger than t' — ► W~b cross section, the analysis for the former 
process has been considered for the remaining of this study. 

Typical kinematical distributions are shown in Figs. 10 and 11. In the event analysis, the signal (f' — > W + b, with k/A = 0.1 
TeV -1 and m t i = 400,500,600 GeV) and background (W + b) are taken into account assuming PHI parametrization. The W- 
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Figure 7: Partonic level pr distributions of b quark from the signal+background process pp — > W + bX at yfs = 14 TeV. The peaks occur around 
respective half mass values of the t' quark masses. 



Table VI: Cross sections for the backgrounds (W b and W j) with pj cuts at the center of mass energy ^fs = 14 TeV. 



Backgrounds 


pr > 20GeV 


p T > 50GeV 


p T > lOOGeV 


W+b(W-b) 


2.79(2.71) 


6.98 x 10 _1 (6-71 x KT 1 ) 


1.16x Kr'(1.09x 10 -1 ) 


W+j(W-j) 


2.22 x 10 4 (1.64x 10 4 ) 


5.37 x 10 3 (3.87x 10 3 ) 


1.02 x 10 3 (6.92x 10 2 ) 



boson invariant mass can be reconstructed from its leptonic or hadronic decays. For the leptonic reconstruction case, the criteria 
is applied to the electrons or muons are: pj > 20 GeV and \r\ l '\ < 2.5 whereas for missing transverse energy, the requirement is 
f.T > 20 GeV. For the hadronic reconstruction case, we require to find at least 2 jets with p J T > 20 GeV and \t] j \ < 2.5. For the 

reconstruction of the t' quark invariant mass, the ^-tagged jets are required to have pj > 50 GeV and |tj ; *| < 2 for both cases. 

The final results are presented only for the case where the W boson is reconstructed from its leptonic decays. The four 
momentum vector of the neutrino is calculated from lepton and missing transverse energy assuming a W boson rest mass 
constraint. The plots for the reconstucted f' invariant mass after detector simulation is given in Fig. 12 at ^fs = 14 TeV and in 
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Figure 8: Invariant mass distributions of b£v system at partonic level for PI (with pj ! > 100 GeV), PII (with p'j' > 50 GeV), and PHI (with 
p b j^ > 50 GeV) parametrizations at *fs =14 TeV. 

Fig. 13 at ^/s = 10 TeV. We include the possible backgrounds contributing to the same final state, and count the signal (5) and 
background (B) events in the corresponding mass intervals to calculate the statistical significance (55) defined as [32] : 



55 = 



(5+B)ln(l + -)-5 



(14) 



In Table VII, the significance calculations are presented for different mass and anomalous coupling values at ^/s =10 TeV. 
Here, we use the mass bin width Am = max(2r, dm) to count signal and background events with the mass resolution dm. The 
significance increases with k/A assuming a maximal mixing between the fourth and the third family quarks. The results of 
this study show that, with early LHC data, one can discover extra up-type quark if there is large anomalous coupling with other 
up-type quarks. 
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Figure 9: Transverse momentum of the i-jet versus W + b invariant mass at ^/s = 14 TeV, on the upper panel only the background distribution 
with W + b (black) and W + j(b) (cyan) events is shown, while on the lower panel both the background and signal distribution with mass of 350 
(red), 550 (green) and 750 GeV (blue). Here, the anomalous coupling is taken to be k/A = 1 TeV -1 and the mixing with the fourth family 
quark t' is parametrized as the PII. 




Figure 10: The pj distributions of jets, electrons and muons from the signal (?' — > W + b) on the left panel, and background (W + b) on the right 
panel shown after detector simulation. The jets are ordered according to the magnitude of their transverse momenta, here we apply the cuts 
p(' > 50 GeV, p J f } > 20 GeV, p' f > 20GeV and fi T > 20 GeV. 
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Figure 11: The r\ distributions of jets, electrons, muons from the simulated signal (/' — > W + b) on the left panel, and background (W + b) on 
the right panel. The jets are ordered according to the magnitude of their pj. Typical LHC detectors have the acceptance for \ r\\ < 2.5 when 
utilization of inner tracker is imposed. 
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Figure 12: Invariant mass distribution of b£v system for both signal (if /A = 0. 1 TeV 1 ) and background at y/s = 14 TeV. 



Table VII: The statistical significance (55) for different masses of t 1 quark, where we take some anomalous parameters in the range (if = 
0.1 - 1.0) at -Js = 10 TeV and L int = 100 pb" 1 . 



55 


m t < = 400 GeV 


m,' = 500 GeV 


m f = 600 GeV 


ff/A = 0.1 TeV" 1 


1.91 


1.99 


2.25 


ff/A^O^TeV- 1 


6.95 


7.19 


8.11 


ff/A = 0.4TeV-' 


21.09 


21.40 


23.44 


ff/A^O.eTeV- 1 


41.97 


38.51 


38.28 


if/A = 0.8 TeV~' 


55.05 


53.05 


50.10 


jc/A= 1 TeV" 1 


65.22 


63.14 


59.00 
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Figure 13: Mass reconstruction of the /' signal (Mv) (solid) with k/A = 0.4 TeV 1 for m,/ = 400 GeV (upper-left), m,< = 500 GeV (upper- 
right) and m,i = 600 GeV (lower), and the corresponding background (dashed) at = 10 TeV. 



VI. CONCLUSION 



Anomalous interactions could become significant at tree level processes due to possible large mass of the fourth family quarks. 
The fourth family t ' quarks can be produced with large numbers if they have anomalous couplings dominates over the SM chiral 
interactions. Following the results from the signal significance for t' anomalous production the sensitivity to anomalous coupling 
k/A can be reached down to 0.1 TeV~'in the b-jet+lepton+^Y channel at ^/s =10 TeV assuming a maximal parametrization 
for the extended CKM elements. The LHC experiments can observe the fourth family quarks mostly in pairs and single in the 
^-channel if they have large anomalous couplings to the known quarks. If detected at the LHC experiments the fourth family 
quarks will change our perpective on the flavor and the mass. 
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